Window - based Data Processing with Stratosphere

Analyzing large amounts of ordered data is a common task in research and industry. The usual ordering domain is time: Examples for time-ordered data are sensor data, communication network data, or financial data. Besides online monitoring, it is common to investigate patterns or special events in the data after capturing it. These analysis can traditionally be performed within Data Stream Management Systems or Relational Database Management Systems. We decided to use the parallelization framework Stratosphere: By design, Stratosphere provides scalability by using clusters or clouds for computations. For ordered data analysis, sliding window semantics are necessary, which are not yet included within the operators of Stratosphere. In this work, we describe sliding window semantics from streaming databases and define Stratosphere operators with sliding window semantics. We introduce an exemplary implementation of one sliding window operator and evaluate its performance. The results show that Stratosphere with sliding window operators is a good choice for analysis on large amounts of ordered data. Augmented with the proposed sliding window operators, the applicability of Stratosphere gets broadened towards an even more general-purpose parallelization framework.